Systems And Methods For Providing Real-Time Composite Video From Multiple Source Devices

ABSTRACT

Systems and methods for superimposing the human elements of video generated by computing devices, wherein a first user device and second user device capture and transmit video to a central server which analyzes the video to identify human elements, superimposes these human elements upon one another, and transmits the newly created superimposed video back to at least one of the user devices.

BACKGROUND OF THE INVENTION

The present subject matter relates generally to a teleconferencingsystem. More specifically, the present invention relates toteleconferencing system that simulates the mutual physical presence ofusers in a virtual interaction.

A phone call over almost any distance is an inherently impersonalexperience. Many developments in recent technology have aimed to solvethe issue of people missing the aspects of human interactions hearingsomeone's voice alone does not provide. From teleconferencing, toFacetime and Snapchat, the use of video calling and messages has greatlyenhanced communicating over great distances, but these innovations arenot without their shortcomings.

Existing video call technology does not allow a user to feel as thoughthey are in close proximity to the person being called. While users maybe able to see and speak with a colleague or loved one over Facetime,etc. it is readily apparent both users of such technology are inseparate locations.

Accordingly, there is a need for a video communication system thatsimulates the mutual physical presence of users in a virtualinteraction.

BRIEF SUMMARY OF THE INVENTION

To meet the needs described above and others, in one embodiment, thesubject matter provided herein is embodied in a video call applicationthat provides users the illusion of both being present in a singlephysical location. Specifically, the embodiment presents the users thevisual experience of being able to reach out and touch the person withwhom they are speaking. The experience is provided through anapplication that allows users to make a video call with the additionalbenefit of being able to superimpose the video from other user(s)devices' cameras onto the video displayed on each of the user devices'screens. This can be used to provide a real time video image of allcallers on the same screen, simulating that everyone involved in thecall are physically in each other's presence.

The application may be standalone or integrated into other video callingapplications. The application may run on mobile devices (e.g.,smartphones, tablets, etc.) and personal computers (e.g., desktopcomputers, laptops, etc.).

One way in which the application may achieve the superimposed videoeffect is by use of the multiple cameras of a smartphone or tablet. Mostmobile devices have two cameras, one on the front face of the device andone on the back. Some newer devices (e.g., the iPhone 7) includemultiple cameras on the back of the device in addition to one or morefront facing cameras. In a first example, the application may allowmultiple cameras on a user device to be active at the same time, withthe system superimposing the human elements (e.g., face, body, hands,etc.) of the video captured by device cameras to give an illusion ofphysical interaction.

In a second example, the application may utilize a first user's frontfacing camera and a second user's rear facing camera to enable the firstuser to physically reach around to the back of the first user devicesuch that the first user's hand (a human element of a video) appears onthe second user's screen when the first user's hand is in view of theirdevice's back camera. The first user's hand could be superimposed overthe face, body, or any other human (or non-human) element(s) captured bythe second user's camera. This allows users of the system to carry outthe illusion of physical interactions such as shaking hands,high-fiving, etc. depending on which device cameras are utilized by endusers.

The video from all cameras utilized by system users at a given time maybe fed into a central server, which in turn transmits the video(s) toother user(s) involved in a given video call. The transmission andreception of the video calls may be carried out via the internet or anyother functionally capable communications network with thesuperimposition of video carried out by user devices, the centralserver, or both depending on what is most functionally advantageous.

The application may also allow users to create a user profile whichfeatures information about the user, their call preferences, contacts,etc. User profiles may be stored in the memory of the central server, onuser devices, or both.

The application may allow for many different video call modes,including: Traditional Video Call—front or rear facing camera only; OneWay Touch Call—a superimposed real time video image of one user's frontcamera and another user's rear camera (or vice versa); Rear Touch Call—asuperimposed real time video image of both users' rear cameras(typically used to show holding hands, etc.); and Front Touch Call—asuperimposed real time video image of both users' front cameras(typically used to show a kiss, etc.).

A goal of the present invention is to increase the quality and intimacyof video calls. By using the front and rear camera's on a smartphone/tablet, the video superimposing software gives the impression ofreaching out and touching another person, shaking hands, kissing, etc.Such interactions are not possible with traditional video chat and couldbe invaluable to long distance couples, isolated elderly people,overseas business partners, etc.

In one embodiment, a computer-implemented method of superimposing videocarried out by a processor, the method includes the steps of: receivinga first live video from a first user device; receiving a second livevideo from a second user device; identifying a first human element inthe first live video and a second human element in the second livevideo; combining a portion of the first live video and a portion of thesecond live video in real-time to create a superimposed video includinga frame perimeter within which a combined portion of the first livevideo and second live video is contained, wherein the superimposed videoincludes the first human element and the second human element, wherein,within the superimposed video, the first human element and the secondhuman element may concurrently occupy any location within the frameperimeter; and transmitting the superimposed video to at least one ofthe first user device and the second user device.

In some examples, in response to real-time movement by the first humanelement in the first live video and the second human element in thesecond live video, contact is simulated between the first human elementand the second human element in the superimposed video.

In other examples, in response to real-time movement by the first humanelement in the first live video and the second human element in thesecond live video, the first human element is superimposed upon thesecond human element in the superimposed video such that the first humanelement obscures at least a portion of the second human element.

In some examples, the first user device is a mobile computing device, inothers, it is a personal computer. In some examples, the first livevideo is captured by a camera of the first user device. In otherexamples, the first live video is captured by at least two cameras ofthe first user device simultaneously.

In other examples, the first live video is captured by a rear facingcamera of the first user device, the second live video is captured by afront facing camera of the second user device, and the first humanelement obscures at least a portion of the second human element in thetransmitted superimposed video. In still others, the first live video iscaptured by a rear facing camera of the first user device, the secondlive video is captured by a rear facing camera of the second userdevice, and the first human element obscures at least a portion of thesecond human element in the transmitted superimposed video.Alternatively, the first live video may be captured by a front facingcamera of the first user device, the second live video is captured by afront facing camera of the second user device, and the first humanelement obscures at least a portion of the second human element in thetransmitted superimposed video.

Yet another embodiment may feature a computer-implemented system forsuperimposing video, comprising: a central server featuring a processor,memory, and networking interface; a first user device featuring acamera, processor, memory, and networking interface; a second userdevice featuring a camera, processor, memory, and networking interface;wherein the central server, receives a first live video from a firstuser device and a second live video from a second user device,identifies a first human element in the first video and a second humanelement in the second video, combines a portion of the first video and aportion of the second video in real-time to create a superimposed videoincluding a frame perimeter within which a combined portion of the firstvideo and second video is contained, wherein the superimposed videoincludes the first human element and the second human element, wherein,within the superimposed video, the first human element and the secondhuman element may concurrently occupy any location within the frameperimeter, and transmits the superimposed video to at least one of thefirst user device and the second user device.

This system may, in response to real-time movement by the first humanelement in the first live video and the second human element in thesecond live video, contact is simulated between the first human elementand the second human element in the superimposed video. The system mayalso, in response to real-time movement by the first human element inthe first live video and the second human element in the second livevideo, the first human element is superimposed upon the second humanelement in the superimposed video such that the first human elementobscures at least a portion of the second human element.

The system may run on a smartphone or desktop computer, wherein thefirst live video is captured by a rear facing camera of the first userdevice, the second live video is captured by a front facing camera ofthe second user device, and the first human element obscures at least aportion of the second human element in the transmitted superimposedvideo. Alternatively, the first live video may be captured by a rearfacing camera of the first user device, the second live video iscaptured by a rear facing camera of the second user device, and thefirst human element obscures at least a portion of the second humanelement in the transmitted superimposed video. Additionally, the firstlive video may be captured by a front facing camera of the first userdevice, the second live video is captured by a front facing camera ofthe second user device, and the first human element obscures at least aportion of the second human element in the transmitted superimposedvideo.

In some examples of the systems and methods described herein, thesuperimposed video may simply be the human element of both callers' livevideo feeds superimposed together. In another example, it may be thehuman element of one caller's live video feed superimposed over the fulllive video feed from the second caller. It is contemplated that there isa technical advantage to just overlaying one identified human element,rather than selecting two human elements. For example, just overlayingone identified human element over the full video feed of the othercaller requires less computing resources and ideally results in lesslatency.

An advantage of the present invention is that the application givesanother dimension to traditional video calls and allows friends andfamilies that are apart from each other to experience the sensation ofbeing able to touch their loved ones from anywhere with an internetconnection. The present invention could allow someone climbing Mt.Everest to call someone in the depths of the Amazon rainforest and bothparties could simulate being beside one another and able to touch as ifthey were side-by-side.

Additional objects, advantages and novel features of the examples willbe set forth in part in the description which follows, and in part willbecome apparent to those skilled in the art upon examination of thefollowing description and the accompanying drawings or may be learned byproduction or operation of the examples. The objects and advantages ofthe concepts may be realized and attained by means of the methodologies,instrumentalities and combinations particularly pointed out in theappended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawing figures depict one or more implementations in accord withthe present concepts, by way of example only, not by way of limitations.In the figures, like reference numerals refer to the same or similarelements.

FIG. 1 is a schematic diagram of a multi-feed video call system.

FIG. 2 is a schematic diagram illustrating an example of an end userdevice that may be used in the system shown in FIG. 1.

FIG. 3 is a flowchart illustrating a computer-implemented method ofsuperimposing video carried out by a processor.

FIG. 4 illustrates the multi-feed video call system superimposingvideos.

FIG. 5A illustrates a One Way Touch call using the multi-feed video callsystem.

FIG. 5B is a hybrid schematic-flowchart showing how the components ofthe multi-feed video call system may cooperate to accomplish a One WayTouch call using the process shown in FIG. 3.

FIG. 5C illustrates a user device displaying a superimposed video in aOne Way Touch call.

FIG. 6A illustrates a Front Touch call using the multi-feed video callsystem.

FIG. 6B is a hybrid schematic-flowchart showing how the components ofthe multi-feed video call system may cooperate to accomplish a FrontTouch call using the process shown in FIG. 3.

FIG. 7A is an overview diagram of a Rear Touch call using the multi-feedvideo call system.

FIG. 7B is a hybrid schematic-flowchart showing how the components ofthe multi-feed video call system may cooperate to accomplish a RearTouch call using the process shown in FIG. 3.

FIG. 8A is an overview diagram of a Multi-Way Touch call using themulti-feed video call system.

FIG. 8B is a hybrid schematic-flowchart showing how the components ofthe multi-feed video call system may cooperate to accomplish a Multi-WayTouch call using the process shown in FIG. 3.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 is a schematic diagram of a multi-feed video call system 10. Asshown in FIG. 1, the system 10 features multiple end users' devices 20.Each end user device 20 (e.g., a laptop computer, smartphone, tablet,etc.) sends video 210 to a central server 30 from an end user devicecamera subsystem 116 through its wireless communication subsystem(s) 120and receives video 210 from the central server 30 to be displayed andoutput through the end user device I/O subsystem 128 and the end userdevice audio subsystem 124. As shown in FIG. 2, a camera subsystem 116may, for example, include front 118 and back 119 cameras of asmartphone.

As described further herein, a primary object of the system 10 is toenable a portion of a first live video 212 to be superimposed upon asecond live video 214 (illustrated in FIG. 4) to produce a superimposedvideo 310 (illustrated in FIG. 4) in which human elements (e.g., a firsthuman element 216 and second human element 218—also seen in FIG. 4) fromeach of the first live video 212 and the second live video 214 mayinteract and be displayed in any position relative to each other tosimulate the appearance of the human elements from each of the firstlive video 212 and the second live video 214 to be present in the samephysical space.

As shown in FIG. 1, the central server 30 includes a processor 31 andmemory 32 for carrying out the superimposition of video 210 (e.g.,combining portions of a first live video 212 and a second live video 214into the superimposed video 310), as well as a networking interface 33for communication with user devices 20, as described further herein. Thesuperimposed video 310 (shown in FIG. 4) created by the server 30 isthen transmitted back to the user devices 20. The superimposed video 310simulates users being physically in each other's presence.

It should be noted that in the example described above, the analysis,processing, and transformation of video 210 is carried out on thecentral server 30. In alternative embodiments, some, or all, of suchactions may be carried out on one or more of the end user devices 20.

FIG. 2 is a schematic diagram illustrating an example of an end userdevice 20 that may be used in the system shown in FIG. 1. In the exampleshown in FIG. 2, the multi-feed video call system 10 runs as a videoconferencing application embodied in video conferencing software 164 onthe end user device 20. As shown in FIG. 2, the end user device 20 maybea mobile device, such as a smartphone, running video conferencingsoftware 164 to provide the functionality described herein. A user mayinstall the video conferencing software 164 on his or her end userdevice 20 via Apple's App Store, the Android Market, etc. The end userdevice 20 may include a wireless communication subsystem 120 tocommunicate with the central server 30 running the video conferencingsoftware 164.

The user device 20 may include a memory interface 102, controllers 103,such as one or more data processors, image processors and/or centralprocessors, and a peripherals interface 106. The memory interface 102,the one or more controllers 103 and/or the peripherals interface 106 canbe separate components or can be integrated in one or more integratedcircuits. The various components in the user device 20 can be coupled byone or more communication buses or signal lines, as will be recognizedby those skilled in the art.

Sensors, devices, and additional subsystems can be coupled to theperipherals interface 106 to facilitate various functionalities. Forexample, a motion sensor 108 (e.g., a gyroscope), a light sensor 163,and positioning sensors 112 (e.g., GPS receiver, accelerometer) can becoupled to the peripherals interface 106 to facilitate the orientation,lighting, and positioning functions described further herein. Othersensors 114 can also be connected to the peripherals interface 106, suchas a proximity sensor, a temperature sensor, a biometric sensor, orother sensing device, to facilitate related functionalities.

A camera subsystem 116 includes a physical camera (e.g., a chargedcoupled device (CCD) or a complementary metal-oxide semiconductor (CMOS)optical sensor) which can be utilized to facilitate camera functions,such as recording photographs and video clips. Modern smartphones andother devices typically feature more than one physical camera operatedby the camera subsystem 116. Such cameras may be located on the front ofthe device 20—the side of the device with a screen (e.g., front cameras118) or rear of the device 20—the side opposite the screen (e.g., rearfacing cameras 119).

Communication functions can be facilitated through a network interface,such as one or more wireless communication subsystems 120, which caninclude radio frequency receivers and transmitters and/or optical (e.g.,infrared) receivers and transmitters. The specific design andimplementation of the communication subsystem 120 can depend on thecommunication network(s) over which the user device 20 is intended tooperate. For example, the user device 20 can include communicationsubsystems 120 designed to operate over a GSM network, a GPRS network,an EDGE network, a Wi-Fi or Imax network, and a Bluetooth network. Inparticular, the wireless communication subsystems 120 may includehosting protocols such that the user device 20 may be configured as abase station for other wireless devices.

An audio subsystem 122 can be coupled to a speaker 124 and a microphone126 to facilitate voice-enabled functions, such as voice recognition,voice replication, digital recording, and telephony functions.

The I/O subsystem 128 may include a touch screen controller 130 and/orother input controller(s) 132. The touch-screen controller 130 can becoupled to a touch screen 134, such as a touch screen. The touch screen134 and touch screen controller 130 can, for example, detect contact andmovement, or break thereof, using any of a plurality of touchsensitivity technologies, including but not limited to capacitive,resistive, infrared, and surface acoustic wave technologies, as well asother proximity sensor arrays or other elements for determining one ormore points of contact with the touch screen 134. The other inputcontroller(s) 132 can be coupled to other input/control devices 136,such as one or more buttons, rocker switches, thumb-wheel, infraredport, USB port, and/or a pointer device such as a stylus. The one ormore buttons (not shown) can include an up/down button for volumecontrol of the speaker 124 and/or the microphone 126.

The memory interface 102 may be coupled to memory 104. The memory 104can include high-speed random access memory and/or non-volatile memory,such as one or more magnetic disk storage devices, one or more opticalstorage devices, and/or flash memory (e.g., NAND, NOR). The memory 104may store operating system instructions 140, such as Darwin, RTXC,LINUX, UNIX, OS X, iOS, ANDROID, BLACKBERRY OS, BLACKBERRY 10, WINDOWS,or an embedded operating system such as VxWorks. The operating systeminstructions 140 may include instructions for handling basic systemservices and for performing hardware dependent tasks. In someimplementations, the operating system instructions 140 can be a kernel(e.g., UNIX kernel).

The memory 104 may also store communication instructions 142 tofacilitate communicating with one or more additional devices, one ormore computers and/or one or more servers. The memory 104 may includegraphical user interface instructions 144 to facilitate graphic userinterface processing; sensor processing instructions 146 to facilitatesensor-related processing and functions; phone instructions 148 tofacilitate phone-related processes and functions; electronic messaginginstructions 150 to facilitate electronic-messaging related processesand functions; web browsing instructions 152 to facilitate webbrowsing-related processes and functions; media processing instructions154 to facilitate media processing-related processes and functions;GPS/Navigation instructions 156 to facilitate GPS and navigation-relatedprocesses and instructions; camera instructions 158 to facilitatecamera-related processes and functions; and/or other softwareinstructions 160 to facilitate other processes and functions (e.g.,access control management functions, etc.). The memory 104 may alsostore other software instructions controlling other processes andfunctions of the user device 20 as will be recognized by those skilledin the art. In some implementations, the media processing instructions154 are divided into audio processing instructions and video processinginstructions to facilitate audio processing-related processes andfunctions and video processing-related processes and functions,respectively. An activation record and International Mobile EquipmentIdentity (IMEI) 162 or similar hardware identifier can also be stored inmemory 104. As described above, the video conferencing software 164 isalso stored in the memory 104 and run by the controllers 103.

Each of the above identified instructions and applications cancorrespond to a set of instructions for performing one or more functionsdescribed herein. These instructions need not be implemented as separatesoftware programs, procedures, or modules. The memory 104 can includeadditional instructions or fewer instructions. Furthermore, variousfunctions of the user device 20 may be implemented in hardware and/or insoftware, including in one or more signal processing and/or applicationspecific integrated circuits. Accordingly, the user device 20, as shownin FIG. 2, may be adapted to perform any combination of thefunctionality described herein.

Aspects of the systems and methods described herein are controlled byone or more controllers 103. The one or more controllers 103 may beadapted run a variety of application programs, access and store data,including accessing and storing data in associated databases, and enableone or more interactions via the user device 20. Typically, the one ormore controllers 103 are implemented by one or more programmable dataprocessing devices. The hardware elements, operating systems, andprogramming languages of such devices are conventional in nature, and itis presumed that those skilled in the art are adequately familiartherewith.

For example, the one or more controllers 103 may be a PC basedimplementation of a central control processing system utilizing acentral processing unit (CPU), memories and an interconnect bus. The CPUmay contain a single microprocessor, or it may contain a plurality ofmicrocontrollers 103 for configuring the CPU as a multi-processorsystem. The memories include a main memory, such as a dynamic randomaccess memory (DRAM) and cache, as well as a read only memory, such as aPROM, EPROM, FLASH-EPROM, or the like. The system may also include anyform of volatile or non-volatile memory. In operation, the main memoryis non-transitory and stores at least portions of instructions forexecution by the CPU and data for processing in accord with the executedinstructions.

The one or more controllers 103 may further include appropriateinput/output ports for interconnection with one or more output displays(e.g., monitors, printers, touchscreen 134, motion-sensing input device108, etc.) and one or more input mechanisms (e.g., keyboard, mouse,voice, touch, bioelectric devices, magnetic reader, RFID reader, barcodereader, touchscreen 134, motion-sensing input device 108, etc.) servingas one or more user interfaces for the processor. For example, the oneor more controllers 103 may include a graphics subsystem to drive theoutput display. The links of the peripherals to the system may be wiredconnections or use wireless communications.

Although summarized above as a smartphone-type implementation, thoseskilled in the art will recognize that the one or more controllers 103also encompasses systems such as host computers, servers, workstations,network terminals, PCs, and the like. Further one or more controllers103 may be embodied in a user device 20, such as a mobile electronicdevice, like a smartphone or tablet computer. In fact, the use of theterm controller is intended to represent a broad category of componentsthat are well known in the art.

FIG. 3 is a flowchart illustrating a computer-implemented method ofsuperimposing video 210 carried out by a processor 31. As shown in FIG.3, the method of superimposing video 210 carried out by a processor 31begins with the processor 31, at a first step 240 receiving a first livevideo 212 from a first user's device 20. Reception 240 by a processor 31is illustrated in FIG. 4, wherein the user device 20 of a first usertransmits a first live video 212 (in this case a video 210 captured bythe user's rear camera 119) to a processor 31 containing central server30. The second step 242 of superimposing video 210 carried out by aprocessor 31 is receiving a second live video 214 from a second user'sdevice 20. Again referring to FIG. 4, reception of the second live video214 from a second user's device 20 by a processor 31 is illustrated(with the second live video 214 being captured by the second user's rearcamera 119). The third step 244 of this method calls for the processor31 to identify a first human element 216 in the first video 212 and asecond human element 218 in a second video 214. Such human elements 216,218 are illustrated in FIG. 4 with the first human element 216 being ahand (captured by the first user's rear camera 119) and the second humanelement 218 being a face (captured by the second user's front camera118).

The fourth step 246 of the computer-implemented method of superimposingvideo 210 carried out by a processor 31 is combing a portion of thefirst video 213 and a portion of the second video 215 in real-time tocreate a superimposed video 310 including a frame perimeter 309 withinwhich a combined portion of the first video 212 and second video 214 iscontained, wherein the superimposed video 310 includes the first humanelement 216 and the second human element 218, wherein, within thesuperimposed video 310, the first human element 216 and the second humanelement 218 may concurrently occupy any location within the frameperimeter 309.

A key feature of the multi-feed video call system 10 is that, within thesuperimposed video 310, the first human element 216 and the second humanelement 218, are each able to occupy any portion of the superimposedvideo 310. In the example shown in FIG. 4, this feature is representedby the ability of either the first human element 216 or the second humanelement 218 to occupy any space within the frame perimeter 309 and theframe perimeter 309 is shown to occupy the entirety of the display onthe device 20. While this is one contemplated example, it is understoodthat the frame perimeter 309 for the superimposed video 310 mayotherwise occupy a smaller portion of the display on the device 20. Thecritical concept relating to the ability of either the first humanelement 216 or the second human element 218 to occupy any space withinthe frame perimeter 309 is that the first human element 216 and thesecond human element 218 may occupy adjacent positions on the screen,may separate from each other, and may pass in front of or behind eachother, or any be represented by any combination of these relativepositions. For example, some portion of the first human element 216 maybe shown to be touching the second human element 218 while otherportions of the first human element 216 may be shown to be separatedfrom the second human element 218.

The portion of the first video 213 and a portion of the second video 215combined in real-time may be only the first human element 216 and secondhuman element 218, or can include more video 210 from the live videos212, 213. This additional video 210, beyond the human elements 216, 218may include a background 219. Such a background 219 is illustrated inFIG. 4 (in this case a plain solid color), with the human elements 216,218 able to occupy any space within the frame perimeter 309 and moveindependently of each other and the background 219. The background 219can be generated by the system 10 of captured by a device 20 camera 118,119.In the example shown in FIG. 4, the first human element 216 (a hand)is superimposed on top of the second human element 218 (a face) withboth elements 216, 218 being able to occupy the same location at thesame time. In this example, since the first human element 216 issuperimposed by the processor 31 over the second human element 218, thehand is able to obscure the face in the superimposed video 310. Theframe perimeter 309, also illustrated in FIG. 4, is the defined size ofthe superimposed video 310 (e.g., the aspect ratio, etc.) which may beautomatically determined by the processor 31 based off the videos 210provided to it.

The final step 248 of the computer-implemented method of superimposingvideo 210 carried out by a processor 31 is transmitting the superimposedvideo 310 to a user device 20. Such transmission is shown in FIG. 4, inwhich the superimposed video 310 is displayed on the first user andsecond user's devices 20. Worth noting here is that the example shown inFIG. 4 utilizes one user device's 20 front camera 118 and one userdevice's 20 back camera 119, but the present invention may also utilizemultiple cameras of a device 20 facing the same direction to capturevideo 210 concurrently. One such device 20 with multiple cameras facingthe same direction is the iPhone 7, which is capable of blending orstitching images captured by is multiple cameras together to providewide angle images, greater image quality, etc. Such functionality may beutilized by the current invention to improve the quality of video callscarried out upon it.

Additionally, FIG. 4 demonstrates two human elements 216, 218 which mayoccupy the same location within a frame perimeter 309 at the same time.This results in one of the human elements 216 being able to obscure theother 218. The present system 10 may also be configured in a mannerwhich prevents the identified human elements 216, 218 from obscuring oneanother. In practical terms, the human elements 216, 218 would betreated as “solid” instead of one element 216 being able to pass overand obscure another 218, with the background 219, frame perimeter 309,etc. being automatically adjusted by the system 10 to prevent suchoverlap.

FIG. 5A is an overview diagram of a One Way Touch call using themulti-feed video call system 10. As shown in FIG. 5A, a superimposedreal time video 310 of one user's front camera 118 (Caller 2) andanother user's rear camera 119 (Caller 1) is displayed to both users. Inthis example, the video 210 from Caller 2's device's front camera 118 isshown on both Caller 1 and Caller 2's device 20 screens, with the video210 from Caller 1's device's back camera 119 superimposed over Caller2's front camera 118 video 210 allowing Caller 1 to “touch” (i.e., seetheir hand or other human element(s) superimposed upon the face and/orbody of another user) Caller 2 via an illusion created by thesuperimposition of the video 210. In this example, the first humanelement 216 is associated with Caller 1's hand and the second humanelement 218 is associated with Caller 2's face, neck, and upper torso.It should be noted however the labels regarding the first human element216 and second human element 218 could be reversed in this example (andthe examples seen in FIGS. 6A, 7A, and 8A) as which human element islabeled first and second does not automatically dictate which elementwill be superimposed over the other.

FIG. 5B is a flowchart of a One Way Touch call using the multi-feedvideo call system 10. As shown in FIG. 5B, the video 210 which will besuperimposed by the system 10 originate on respective caller's end userdevices 20. One user's (Caller 1's) rear camera 119 and another user's(Caller 2's) front camera 118 send video 210 to a centralized sever 30.In this embodiment, as well as the embodiments illustrated in FIGS. 3C,4B, 5B, and 6B, the system 10 may automatically determine which user'svideo 210 is superimposed over the other (e.g., which human element(hand, face, torso, etc.) is superimposed over the other humanelement(s) displayed in the superimposed video 310). In otherembodiments, the determination of which human element(s) of video 210are superimposed upon one another may also be manually set by theparticipants of a given video call or the system 10 may be set to notsuperimpose human elements. The video 210 may be sent via the internetor any other functionally useful means, with the central server 30receiving the separate video 210, analyzing them, removing extraneousinformation from the video 210 (e.g., solid colored backgrounds, etc.),and combining the two respective video 210 into one superimposed video310. The superimposed video 310 is then sent back to the user device's20 involved in the video chat via the internet or any other functionallyuseful means.

In this example, the first live video 212 is associated with Caller 1'sand the second live video 214 is associated with Caller 2. It should benoted however the labels regarding the live videos 212, 214 could bereversed in this example (and the examples seen in FIGS. 6B, 7B, and 8B)as which video 212, 214 is labeled first and second does notautomatically dictate which will be superimposed over the other.

FIG. 5C is a diagram of a user device 20 displaying a One Way Touchcall. As shown in FIG. 5C, an end user device 20 may display a superimposed video 310 which features, in this example, the video 210 fromCaller 2's (as shown in FIG. 3A) device's front camera 118 with thevideo 210 from Caller 1's (as shown in FIG. 3A) device's back camera 119superimposed over Caller 2's front camera 118 video 210 allowing Caller1 to “touch” (i.e., see their hand or other human element(s)superimposed upon the face and/or body of another user) Caller 2 via anillusion shown within the superimposed video 310.

FIG. 6A is an overview diagram of a Front Touch call using themulti-feed video call system 10. As shown in FIG. 6A, a superimposedreal time video 310 of both users' front camera 118 (Caller 1 and Caller2) is displayed to both users. In this example, the video 210 fromCaller 1's device's front camera 118 is shown on both Caller 1 andCaller 2's device 20 screens, with the video 210 from Caller 2'sdevice's front camera 118 superimposed over Caller 2's front camera 118video 210, allowing the users to appear to be physically side by side.

FIG. 6B is a flowchart of a Front Touch call using the multi-feed videocall system 10. As shown in FIG. 6B, the video 210 which will besuperimposed by the system 10 originate on respective caller's end userdevices 20. Both users' (Caller 1 and Caller 2) front camera 118 sendvideo 210 to a centralized server 30. The video 210 may be sent via theinternet or any other functionally useful means, with the central server30 receiving the separate video 210, analyzing them, removing extraneousinformation from the video 210 (e.g., solid colored backgrounds, etc.),and combining the two respective video 210 into one superimposed video310. The superimposed video 310 is then sent back to the user device's20 involved in the video chat via the internet or any other functionallyuseful means.

FIG. 7A is an overview diagram of a Rear Touch call using the multi-feedvideo call system 10. As shown in FIG. 7A, a superimposed real timevideo 310 of both users' rear cameras 119 (Caller 1 and Caller 2) isdisplayed to both users. In this example, the video 210 from Caller 1'sdevice's rear camera is shown on both Caller 1 and Caller 2's device 20screens, with the video 210 from Caller 2's device's rear camera 119superimposed over Caller 1's rear camera video 210, forming thesuperimposed video 310, and allowing the users to appear to bephysically holding hands, etc.

FIG. 7B is a flowchart of a Rear Touch call using the multi-feed videocall system 10. As shown in FIG. 7B, the video 210 which will besuperimposed by the system 10 originate on respective caller's end userdevices 20. Both users' (Caller 1 and Caller 2) rear camera 119 sendvideo 210 to a centralized server 30. The video 210 may be sent via theinternet or any other functionally useful means, with the central server30 receiving the separate video 210, analyzing them, removing extraneousinformation from the videos 210 (e.g., solid colored backgrounds, etc.),and combining the two respective video 210 into one superimposed video310. The superimposed video 310 is then sent back to the user device's20 involved in the video chat via the internet or any other functionallyuseful means.

FIG. 8A is an overview diagram of a Multi-Way Touch call using themulti-feed video call system 10. As shown in FIG. 8A, a superimposedreal time video 310 of a first user's front camera 118 (Caller 1) and asecond user's rear camera 119 (Caller 2) is displayed to the seconduser, with a superimposed real time video 310 of the video of the seconduser's front camera 118 (Caller 2) and the first user's rear camera 119(Caller 1) displayed to the first user. This allows both users to“touch” the other user simultaneously with the visual effect beingenabled by the superimposition of video.

FIG. 8B is a flowchart of a Multi-Way Touch call using the multi-feedvideo call system 10. As shown in FIG. 8B, the video 210 which will besuperimposed by the system 10 originate on respective caller's end userdevices 20. Both user's rear camera 119 and front camera 118 send video210 to a centralized server 30. The video may be sent via the internetor any other functionally useful means, with the central server 30receiving the separate video 210, analyzing them, removing extraneousinformation from the video 210 (e.g., solid colored backgrounds, etc.),and combining the four respective video 210 into two superimposed video310 (as discussed in FIG. 8A). The superimposed video 310 are then sentback to the respective user device's 20 involved in the video chat viathe internet or any other functionally useful means.

Hence aspects of the systems and methods provided herein encompasshardware and software for controlling the relevant functions. Softwaremay take the form of code or executable instructions for causing aprocessor or other programmable equipment to perform the relevant steps,where the code or instructions are carried by or otherwise embodied in amedium readable by the processor or other machine. Instructions or codefor implementing such operations may be in the form of computerinstruction in any form (e.g., source code, object code, interpretedcode, etc.) stored in or carried by any tangible readable medium.

It should be noted that various changes and modifications to thepresently preferred embodiments described herein will be apparent tothose skilled in the art. Such changes and modifications may be madewithout departing from the spirit and scope of the present invention andwithout diminishing its attendant advantages.

1. A computer-implemented method of superimposing video carried out by aprocessor, the method comprising the steps of: receiving a first livevideo from a first user device; receiving a second live video from asecond user device; extracting a first human element from the first livevideo; combining the first human element and a portion or all of thesecond live video in real-time to create a superimposed video includinga frame perimeter within which the superimposed video includes the firsthuman element and a second human element from the second live video,wherein, within the superimposed video, the first human element and thesecond human element may concurrently occupy any location within theframe perimeter; and transmitting the superimposed video to at least oneof the first user device and the second user device; wherein the firstlive video is captured by a rear facing camera of the first user deviceand, in response to movement of the first human element relative to thefirst user device, the first human element obscures at least a portionof the second human element in the transmitted superimposed videofurther wherein the location of the extracted human element within thesuperimposed video is directly controlled in real-time by the positionof the first human element relative to the first user device.
 2. Thecomputer-implemented method of superimposing video of claim 1, whereinin response to real-time movement by the first human element relative tothe first user device and the second human element relative to thesecond user device, contact is simulated between the first human elementand the second human element in the superimposed video.
 3. Thecomputer-implemented method of superimposing video of claim 1, whereinthe first user device is a mobile computing device.
 4. Thecomputer-implemented method of superimposing video of claim 1, whereinthe first user device is a personal computer.
 5. Thecomputer-implemented method of superimposing video of claim 1, whereinthe first live video is captured by at least two cameras of the firstuser device simultaneously.
 6. The computer-implemented method ofsuperimposing video of claim 1, wherein the second live video iscaptured by a front facing camera of the second user device.
 7. Thecomputer-implemented method of superimposing video of claim 1, whereinthe second live video is captured by a rear facing camera of the seconduser device.
 8. A computer-implemented system for superimposing video,comprising: a central server featuring a processor, memory, andnetworking interface; a first user device featuring a camera, processor,memory, and networking interface; a second user device featuring acamera, processor, memory, and networking interface; wherein the centralserver: receives a first live video from a first user device and asecond live video from a second user device; extracts a first humanelement from the first live video; combines the first human element witha portion or all of the second live video in real-time to create asuperimposed video including a frame perimeter within which thesuperimposed video includes the first human element and a second humanelement from the second live video, wherein, within the superimposedvideo, the first human element and the second human element mayconcurrently occupy any location within the frame perimeter; andtransmits the superimposed video to at least one of the first userdevice and the second user device; wherein the first live video iscaptured by a rear facing camera of the first user device and, inresponse to movement of the first human element relative to the firstuser device, the first human element obscures at least a portion of thesecond human element in the transmitted superimposed video furtherwherein the location of the extracted human element within thesuperimposed video is directly controlled in real-time by the positionof the first human element relative to the first user device.
 9. Thecomputer-implemented system for superimposing video of claim 8, whereinin response to real-time movement by the first human element relative tothe first user device and the second human element relative to thesecond user device, contact is simulated between the first human elementand the second human element in the superimposed video.
 10. Thecomputer-implemented system for superimposing video of claim 8, whereinthe first user device is a smartphone.
 11. The computer-implementedsystem for superimposing video of claim 8, wherein the first user deviceis a desktop computer.
 12. The computer-implemented system ofsuperimposing video of claim 8, wherein the first live video is capturedby at least two cameras of the first user device simultaneously.
 13. Thecomputer-implemented system for superimposing video of claim 8, whereinthe second live video is captured by a front facing camera of the seconduser device.
 14. The computer-implemented system for superimposing videoof claim 8, wherein the second live video is captured by a rear facingcamera of the second user device.